Skip to content

[TESTING] Test thrift-remodel branch with datafusion #17916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 5 commits into from
Closed

[TESTING] Test thrift-remodel branch with datafusion #17916

wants to merge 5 commits into from

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Oct 4, 2025 •
edited
Loading

Which issue does this PR close?

Rationale for this change

This PR is not meant to be merged -- I am using it to test the proposed
thrift-remodel branch with DataFusion. The thrift-remodel branch has
a brand new, much faster, thrift metadata decoder.

Specifically, this PR is to run the DataFusion test suite with the thrift-remodel branch
of arrow-rs to see if there are any issues before we merge it.

What changes are included in this PR?

Are these changes tested?

Changes

Things that would help upgrade

  1. Add PrimitiveColumnIndex::len() (can fake with min_values.len())

Are there any user-facing changes?

@github-actions github-actions bot added sql SQL Planner logical-expr Logical plan and expressions physical-expr Changes to the physical-expr crates optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) substrait Changes to the substrait crate common Related to common crate proto Related to proto crate functions Changes to functions implementation physical-plan Changes to the physical-plan crate datasource Changes to the datasource crate labels Oct 4, 2025
@alamb alamb mentioned this pull request Oct 4, 2025
Closed
@alamb alamb force-pushed the alamb/test_thrift_remodel branch from 8909594 to 75f56fc Compare October 6, 2025 20:20
@github-actions github-actions bot removed the sqllogictest SQL Logic Tests (.slt) label Oct 6, 2025
@alamb alamb force-pushed the alamb/test_thrift_remodel branch from 75f56fc to 907b3a2 Compare October 7, 2025 13:58
@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Oct 7, 2025
@alamb
Copy link
Contributor Author

alamb commented Oct 7, 2025

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.14.0-1016-gcp #17~24.04.1-Ubuntu SMP Wed Sep 3 01:55:36 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/test_thrift_remodel (082a22c) to 307f5c3 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

@alamb
Copy link
Contributor Author

alamb commented Oct 7, 2025

🤖: Benchmark completed

Details 

Comparing HEAD and alamb_test_thrift_remodel
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_thrift_remodel ┃    Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 0     │  2700.35 ms │                2710.11 ms │ no change │
│ QQuery 1     │  1314.50 ms │                1314.08 ms │ no change │
│ QQuery 2     │  2377.78 ms │                2478.90 ms │ no change │
│ QQuery 3     │  1200.18 ms │                1212.18 ms │ no change │
│ QQuery 4     │  2234.25 ms │                2231.08 ms │ no change │
│ QQuery 5     │ 27699.83 ms │               27833.86 ms │ no change │
│ QQuery 6     │  4152.07 ms │                4171.78 ms │ no change │
│ QQuery 7     │  3540.75 ms │                3465.12 ms │ no change │
└──────────────┴─────────────┴───────────────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 45219.71ms │
│ Total Time (alamb_test_thrift_remodel)   │ 45417.10ms │
│ Average Time (HEAD)                      │  5652.46ms │
│ Average Time (alamb_test_thrift_remodel) │  5677.14ms │
│ Queries Faster                           │          0 │
│ Queries Slower                           │          0 │
│ Queries with No Change                   │          8 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_test_thrift_remodel ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.17 ms │                   2.25 ms │     no change │
│ QQuery 1     │    49.20 ms │                  47.38 ms │     no change │
│ QQuery 2     │   138.30 ms │                 131.98 ms │     no change │
│ QQuery 3     │   159.35 ms │                 168.28 ms │  1.06x slower │
│ QQuery 4     │  1045.43 ms │                1091.99 ms │     no change │
│ QQuery 5     │  1492.46 ms │                1554.38 ms │     no change │
│ QQuery 6     │     2.13 ms │                   2.12 ms │     no change │
│ QQuery 7     │    53.21 ms │                  53.66 ms │     no change │
│ QQuery 8     │  1434.21 ms │                1473.44 ms │     no change │
│ QQuery 9     │  1838.70 ms │                1845.95 ms │     no change │
│ QQuery 10    │   382.38 ms │                 396.02 ms │     no change │
│ QQuery 11    │   436.88 ms │                 454.99 ms │     no change │
│ QQuery 12    │  1361.72 ms │                1388.66 ms │     no change │
│ QQuery 13    │  2083.61 ms │                2158.60 ms │     no change │
│ QQuery 14    │  1256.48 ms │                1284.43 ms │     no change │
│ QQuery 15    │  1195.69 ms │                1255.00 ms │     no change │
│ QQuery 16    │  2618.19 ms │                2700.84 ms │     no change │
│ QQuery 17    │  2604.96 ms │                2683.48 ms │     no change │
│ QQuery 18    │  5721.59 ms │                5017.27 ms │ +1.14x faster │
│ QQuery 19    │   128.61 ms │                 129.04 ms │     no change │
│ QQuery 20    │  2070.96 ms │                1962.33 ms │ +1.06x faster │
│ QQuery 21    │  2328.83 ms │                2307.45 ms │     no change │
│ QQuery 22    │  4036.89 ms │                3974.83 ms │     no change │
│ QQuery 23    │ 12711.99 ms │               12806.91 ms │     no change │
│ QQuery 24    │   213.13 ms │                 219.26 ms │     no change │
│ QQuery 25    │   514.80 ms │                 521.07 ms │     no change │
│ QQuery 26    │   231.62 ms │                 224.10 ms │     no change │
│ QQuery 27    │  2894.93 ms │                2896.02 ms │     no change │
│ QQuery 28    │ 24837.90 ms │               23176.07 ms │ +1.07x faster │
│ QQuery 29    │   995.76 ms │                 979.67 ms │     no change │
│ QQuery 30    │  1325.19 ms │                1342.25 ms │     no change │
│ QQuery 31    │  1338.85 ms │                1350.07 ms │     no change │
│ QQuery 32    │  4830.25 ms │                4421.62 ms │ +1.09x faster │
│ QQuery 33    │  6069.62 ms │                5566.52 ms │ +1.09x faster │
│ QQuery 34    │  6306.35 ms │                6039.25 ms │     no change │
│ QQuery 35    │  2036.06 ms │                2010.36 ms │     no change │
│ QQuery 36    │   123.25 ms │                 120.80 ms │     no change │
│ QQuery 37    │    53.81 ms │                  53.18 ms │     no change │
│ QQuery 38    │   119.12 ms │                 119.42 ms │     no change │
│ QQuery 39    │   198.46 ms │                 195.11 ms │     no change │
│ QQuery 40    │    45.41 ms │                  44.21 ms │     no change │
│ QQuery 41    │    40.96 ms │                  38.79 ms │ +1.06x faster │
│ QQuery 42    │    32.94 ms │                  32.91 ms │     no change │
└──────────────┴─────────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 97362.32ms │
│ Total Time (alamb_test_thrift_remodel)   │ 94241.96ms │
│ Average Time (HEAD)                      │  2264.24ms │
│ Average Time (alamb_test_thrift_remodel) │  2191.67ms │
│ Queries Faster                           │          6 │
│ Queries Slower                           │          1 │
│ Queries with No Change                   │         36 │
│ Queries with Failure                     │          0 │
└──────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_test_thrift_remodel ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │ 166.71 ms │                 172.26 ms │     no change │
│ QQuery 2     │  26.13 ms │                  26.14 ms │     no change │
│ QQuery 3     │  42.16 ms │                  40.55 ms │     no change │
│ QQuery 4     │  28.04 ms │                  28.23 ms │     no change │
│ QQuery 5     │  79.19 ms │                  75.26 ms │     no change │
│ QQuery 6     │  20.21 ms │                  19.24 ms │     no change │
│ QQuery 7     │ 222.58 ms │                 220.24 ms │     no change │
│ QQuery 8     │  35.17 ms │                  31.38 ms │ +1.12x faster │
│ QQuery 9     │ 107.54 ms │                 104.46 ms │     no change │
│ QQuery 10    │  60.03 ms │                  60.34 ms │     no change │
│ QQuery 11    │  16.97 ms │                  17.73 ms │     no change │
│ QQuery 12    │  51.69 ms │                  51.34 ms │     no change │
│ QQuery 13    │  48.33 ms │                  46.48 ms │     no change │
│ QQuery 14    │  13.83 ms │                  13.78 ms │     no change │
│ QQuery 15    │  24.72 ms │                  23.89 ms │     no change │
│ QQuery 16    │  24.83 ms │                  24.42 ms │     no change │
│ QQuery 17    │ 149.17 ms │                 146.79 ms │     no change │
│ QQuery 18    │ 317.56 ms │                 323.60 ms │     no change │
│ QQuery 19    │  36.65 ms │                  36.25 ms │     no change │
│ QQuery 20    │  48.92 ms │                  49.23 ms │     no change │
│ QQuery 21    │ 333.92 ms │                 320.94 ms │     no change │
│ QQuery 22    │  20.79 ms │                  20.16 ms │     no change │
└──────────────┴───────────┴───────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                        ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                        │ 1875.14ms │
│ Total Time (alamb_test_thrift_remodel)   │ 1852.73ms │
│ Average Time (HEAD)                      │   85.23ms │
│ Average Time (alamb_test_thrift_remodel) │   84.21ms │
│ Queries Faster                           │         1 │
│ Queries Slower                           │         0 │
│ Queries with No Change                   │        21 │
│ Queries with Failure                     │         0 │
└──────────────────────────────────────────┴───────────┘

@alamb
Copy link
Contributor Author

alamb commented Oct 7, 2025

My assessment is that the thrift-remodel branch is ready to go, so closing down this PR: #17916 (comment)

@alamb alamb closed this Oct 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

common Related to common crate core Core DataFusion crate datasource Changes to the datasource crate functions Changes to functions implementation logical-expr Logical plan and expressions optimizer Optimizer rules physical-expr Changes to the physical-expr crates physical-plan Changes to the physical-plan crate proto Related to proto crate sql SQL Planner sqllogictest SQL Logic Tests (.slt) substrait Changes to the substrait crate

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@alamb